Home / Numerical Analysis

Initial-Value Problems for Ordinary Differential Equations

First-order methods

Second-order methods

Higher-order methods

Initial-Value Problems for Ordinary Differential Equations

In this section, we go through several methods that can approximate the solution of a well-posed initial-value problem (IVP) on grid points. Consider the following IVP:

\begin{align} \frac{dy}{dt} &= f(t,y),& a \leq t \leq b,~y(a) = y_{0}. \end{align}

Here it is assumed that the grid points are equally distributed throughout the interval $[a, b]$. Let $N$ be a positive integer, we have, the following step size

\begin{align} h = \Delta t = \frac{b-a}{N} \end{align}

and each grid point can be identified as \begin{align} t_{j} &= a +jh,& j = 0,1,2,\ldots,N. \end{align}

Moreover, let $y_{j}$ denote approximation $y$ at grid points for $j = 1,2,3,\ldots,N$, we can use several methods.

Runge–Kutta methods

In this article, we only discuss explicit methods Runge–Kutta methods. Explicit Runge-Kutta methods take the form \begin{align} y_{n+1} &= y_{n}+h\sum _{i=1}^{s}b_{i}k_{i},\\ k_{i} &= f\left(t_{n}+c_{i}h,y_{n}+h\sum _{j=1}^{i-1}a_{ij}k_{j}\right). \end{align}

This process is carried forward iteratively until point $x_{n-1}$ is reached. To represent the coefficients of this method, the Butcher tableau is often used which puts the coefficients of the method in a table as follows.

\begin{align} \begin{array}{c|cccc} c_1 & a_{11} & a_{12}& \dots & a_{1s}\\ c_2 & a_{21} & a_{22}& \dots & a_{2s}\\ \vdots & \vdots & \vdots& \ddots& \vdots\\ c_s & a_{s1} & a_{s2}& \dots & a_{ss} \\ \hline & b_1 & b_2 & \dots & b_s\\ \end{array} \end{align}

Notes:

Most of the previous explicit methods can be obtained using the Butcher tableau.

Third-order Runge–Kutta method

This method is defined as follows, \begin{align} y_{n+1} &= y_{n}+\frac{h}{6}\left( k_{1} + 4k_{2} + k_{3} \right),\quad \text{for }n = 0, 1, 2, 3, \ldots, \end{align} with \begin{align} \begin{cases} k_{1} & = f\left(t_{n},~y_{n} \right),\\ k_{2} & = f\left(t_{n}+\frac{h}{2},~y_{n}+\frac{h}{2}k_{1} \right),\\ k_{3} & = f\left(t_{n}+h,~y_{n}+ h(2k_{2} -k_{1}) \right) \end{cases} \end{align} or using the Butcher tableau \begin{align}\begin{array}{c|ccc}0&0&0&0\\1/2&1/2&0&0\\1&-1&2&0\\\hline &1/6&2/3&1/6\\\end{array}\end{align}

Furthermore, we can prepare a Python code using the above algorithm.

Example: Conisder the initial value problem $\begin{cases}y'+2ty=te^{-t^2},\\ y(0) = 1 \end{cases},\quad 0 \leq x \leq 1$ with exact solution $y\left(x \right) = \left(1 + \dfrac{t^{2}}{2}\right) e^{- t^{2}}$. We use the the above method for solving this IVP.

For convenience, we can define the following function which can be used for various methods.

Heun's third-order method

This method is defined as follows, \begin{align} y_{n+1} &= y_{n}+\frac{h}{4}\left( k_{1} + 3k_{3} \right) \end{align} with \begin{align} \begin{cases} k_{1} & = f\left(t_{n},~y_{n} \right),\\ k_{2} & = f\left(t_{n}+\frac{h}{3},~y_{n}+\frac{h}{3}k_{1} \right),\\ k_{3} & = f\left(t_{n}+\frac{2}{3}h,~y_{n}+ \frac{2h}{3}k_{2}\right) \end{cases} \end{align} or using the Butcher tableau \begin{align} \begin{array}{c|ccc}0&0&0&0\\1/3&1/3&0&0\\2/3&0&2/3&0\\\hline &1/4&0&3/4\\\end{array} \end{align}

The Butcher tableau can be also generated using nodepy python package. For example, to get the Butcher tableau for the third-order Heun's method, we have,

Furthermore, we can prepare a Python code using the above algorithm.

Example: We can use Heun's third-order method on the previous example IVP.

Ralston's third-order method

This method is defined as follows, \begin{align} y_{n+1} &= y_{n}+\frac{h}{9}\left( 2k_{1} + 3k_{2} + 4k_{3} \right) \end{align} with \begin{align} \begin{cases} k_{1} & = f\left(t_{n},~y_{n} \right),\\ k_{2} & = f\left(t_{n}+\frac{h}{2},~y_{n}+\frac{h}{2}k_{1} \right),\\ k_{3} & = f\left(t_{n}+\frac{3}{4}h,~y_{0} + \frac{3h}{4}k_{2}\right) \end{cases} \end{align} or using the Butcher tableau \begin{align} \begin{array}{c|ccc}0&0&0&0\\1/2&1/2&0&0\\3/4&0&3/4&0\\\hline &2/9&1/3&4/9\\\end{array} \end{align}

Furthermore, we can prepare a Python code using the above algorithm.

Example: We can use Ralston's third-order method on the previous example IVP.

Strong-Stability preserving Runge-Kutta time-steppers (SSPRK3)

This method is defined as follows, \begin{align} y_{n+1} &= y_{n}+\frac{h}{9}\left( 2k_{1} + 3k_{2} + 4k_{3} \right) \end{align} with \begin{align} \begin{cases} k_{1} & = f\left(t_{n},~y_{n} \right),\\ k_{2} & = f\left(t_{n}+h,~y_{n}+hk_{1} \right),\\ k_{3} & = f\left(t_{n}+\frac{h}{2},~y_{0} + \frac{h}{4}k_{1} + \frac{h}{4}k_{2}\right) \end{cases} \end{align} or using the Butcher tableau \begin{align} \begin{array}{c|ccc}0&0&0&0\\1&1&0&0\\1/2&1/4&1/4&0\\\hline &1/6&1/6&2/3\\\end{array} \end{align}

The Butcher tableau can be also generated using nodepy python package. For example, to get the Butcher tableau for the third-order Heun's method, we have,

Furthermore, we can prepare a Python code using the above algorithm.

Example: We can use the SSPRK3 method on the previous example IVP.

References

  1. Allaire, Grégoire, et al. Numerical linear algebra. Vol. 55. New York: Springer, 2008.
  2. Burden, Richard L., and J. Douglas Faires. "Numerical analysis 8th ed." Thomson Brooks/Cole (2005).
  3. Atkinson, Kendall E. An introduction to numerical analysis. John wiley & sons, 2008.
  4. Khoury, Richard, and Douglas Wilhelm Harder. Numerical methods and modelling for engineering. Springer, 2016.
  5. Zarowski, Christopher J. An introduction to numerical analysis for electrical and computer engineers. John Wiley & Sons, 2004.
  6. Iserles, Arieh. A first course in the numerical analysis of differential equations. No. 44. Cambridge university press, 2009.
  7. Runge-Kutta method